{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multi-Horizon Forecasting for Limit Order Books: Novel Deep Learning Approaches and Hardware Acceleration using Intelligent Processing Units\n",
    "### Authors: Zihao Zhang and Stefan Zohren\n",
    "### Oxford-Man Institute of Quantitative Finance, Department of Engineering Science, University of Oxford\n",
    "\n",
    "This jupyter notebook is used to demonstrate our recent paper [2]. We use FI-2010 [1] dataset and present how model architecture is constructed here. The FI-2010 is publicly avilable and interested readers can check out their paper [1]. The dataset can be downloaded from: https://etsin.fairdata.fi/dataset/73eb48d7-4dbc-4a10-a52a-da745b47a649\n",
    "\n",
    "Otherwise, it can be obtained from: https://drive.google.com/drive/folders/1Xen3aRid9ZZhFqJRgEMyETNazk02cNmv?usp=sharing\n",
    "\n",
    "[1] Ntakaris A, Magris M, Kanniainen J, Gabbouj M, Iosifidis A. Benchmark dataset for mid‐price forecasting of limit order book data with machine learning methods. Journal of Forecasting. 2018 Dec;37(8):852-66. https://arxiv.org/abs/1705.03233\n",
    "\n",
    "[2] Zhang Z, Zohren S. Multi-Horizon Forecasting for Limit Order Books: Novel Deep Learning Approaches and Hardware Acceleration using Intelligent Processing Units. https://arxiv.org/abs/2105.10430\n",
    "\n",
    "#### This notebook demonstrates how to train DeepLOB-Attention by using tensorflow 2 on GPUs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 Physical GPUs, 1 Logical GPUs\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "gpus = tf.config.experimental.list_physical_devices('GPU')\n",
    "if gpus:\n",
    "    try:\n",
    "    # Currently, memory growth needs to be the same across GPUs\n",
    "        for gpu in gpus:\n",
    "            tf.config.experimental.set_memory_growth(gpu, True)\n",
    "            logical_gpus = tf.config.experimental.list_logical_devices('GPU')\n",
    "        print(len(gpus), \"Physical GPUs,\", len(logical_gpus), \"Logical GPUs\")\n",
    "    except RuntimeError as e:\n",
    "    # Memory growth must be set before GPUs have been initialized\n",
    "        print(e)\n",
    "\n",
    "# %%\n",
    "import os\n",
    "import logging\n",
    "import glob\n",
    "import argparse\n",
    "import sys\n",
    "import time\n",
    "import pandas as pd\n",
    "import pickle\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from collections import Counter\n",
    "\n",
    "# set random seeds\n",
    "np.random.seed(1)\n",
    "tf.random.set_seed(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from preprocess import *\n",
    "from model_gpu import get_model_attention"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# please change the data_path to your local path\n",
    "data_path = '/nfs/home/zihaoz/limit_order_book/data'\n",
    "T = 50\n",
    "epochs = 50\n",
    "batch_size = 32\n",
    "n_hidden = 64\n",
    "checkpoint_filepath = './model_deeplob_attention/weights'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_encoder_input.shape = (254701, 50, 40, 1),train_decoder_target.shape = (254701, 5, 3)\n",
      "test_encoder_input.shape = (139538, 50, 40, 1),test_decoder_target.shape = (139538, 5, 3)\n"
     ]
    }
   ],
   "source": [
    "# %%\n",
    "dec_train = np.loadtxt(data_path + '/3.NoAuction_DecPre/NoAuction_DecPre_Training/Train_Dst_NoAuction_DecPre_CF_7.txt')\n",
    "dec_test1 = np.loadtxt(data_path + '/3.NoAuction_DecPre/NoAuction_DecPre_Testing/Test_Dst_NoAuction_DecPre_CF_7.txt')\n",
    "dec_test2 = np.loadtxt(data_path + '/3.NoAuction_DecPre/NoAuction_DecPre_Testing/Test_Dst_NoAuction_DecPre_CF_8.txt')\n",
    "dec_test3 = np.loadtxt(data_path + '/3.NoAuction_DecPre/NoAuction_DecPre_Testing/Test_Dst_NoAuction_DecPre_CF_9.txt')\n",
    "dec_test = np.hstack((dec_test1, dec_test2, dec_test3))\n",
    "\n",
    "# extract limit order book data from the FI-2010 dataset\n",
    "train_lob = prepare_x(dec_train)\n",
    "test_lob = prepare_x(dec_test)\n",
    "\n",
    "# extract label from the FI-2010 dataset\n",
    "train_label = get_label(dec_train)\n",
    "test_label = get_label(dec_test)\n",
    "\n",
    "# prepare training data. We feed past 100 observations into our algorithms.\n",
    "train_encoder_input, train_decoder_target = data_classification(train_lob, train_label, T)\n",
    "train_decoder_input = prepare_decoder_input(train_encoder_input, teacher_forcing=False)\n",
    "\n",
    "test_encoder_input, test_decoder_target = data_classification(test_lob, test_label, T)\n",
    "test_decoder_input = prepare_decoder_input(test_encoder_input, teacher_forcing=False)\n",
    "\n",
    "print(f'train_encoder_input.shape = {train_encoder_input.shape},'\n",
    "      f'train_decoder_target.shape = {train_decoder_target.shape}')\n",
    "print(f'test_encoder_input.shape = {test_encoder_input.shape},'\n",
    "      f'test_decoder_target.shape = {test_decoder_target.shape}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = get_model_attention(n_hidden)\n",
    "model.compile(loss='categorical_crossentropy', metrics=['accuracy'], optimizer='adam')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "6368/6368 - 220s - loss: 0.9544 - accuracy: 0.5053 - val_loss: 0.9682 - val_accuracy: 0.5224\n",
      "Epoch 2/50\n",
      "6368/6368 - 218s - loss: 0.7863 - accuracy: 0.6332 - val_loss: 0.8159 - val_accuracy: 0.6323\n",
      "Epoch 3/50\n",
      "6368/6368 - 218s - loss: 0.6645 - accuracy: 0.7234 - val_loss: 0.7634 - val_accuracy: 0.6678\n",
      "Epoch 4/50\n",
      "6368/6368 - 218s - loss: 0.6199 - accuracy: 0.7456 - val_loss: 0.7506 - val_accuracy: 0.6822\n",
      "Epoch 5/50\n",
      "6368/6368 - 219s - loss: 0.5961 - accuracy: 0.7569 - val_loss: 0.8039 - val_accuracy: 0.6706\n",
      "Epoch 6/50\n",
      "6368/6368 - 219s - loss: 0.5829 - accuracy: 0.7624 - val_loss: 0.7112 - val_accuracy: 0.7020\n",
      "Epoch 7/50\n",
      "6368/6368 - 218s - loss: 0.5722 - accuracy: 0.7673 - val_loss: 0.7078 - val_accuracy: 0.7006\n",
      "Epoch 8/50\n",
      "6368/6368 - 218s - loss: 0.5640 - accuracy: 0.7709 - val_loss: 0.7267 - val_accuracy: 0.6994\n",
      "Epoch 9/50\n",
      "6368/6368 - 217s - loss: 0.5572 - accuracy: 0.7737 - val_loss: 0.7225 - val_accuracy: 0.6986\n",
      "Epoch 10/50\n",
      "6368/6368 - 218s - loss: 0.5518 - accuracy: 0.7758 - val_loss: 0.7264 - val_accuracy: 0.7018\n",
      "Epoch 11/50\n",
      "6368/6368 - 217s - loss: 0.5472 - accuracy: 0.7781 - val_loss: 0.7319 - val_accuracy: 0.6975\n",
      "Epoch 12/50\n",
      "6368/6368 - 217s - loss: 0.5440 - accuracy: 0.7793 - val_loss: 0.7277 - val_accuracy: 0.7036\n",
      "Epoch 13/50\n",
      "6368/6368 - 217s - loss: 0.5403 - accuracy: 0.7810 - val_loss: 0.7253 - val_accuracy: 0.7044\n",
      "Epoch 14/50\n",
      "6368/6368 - 217s - loss: 0.5396 - accuracy: 0.7811 - val_loss: 0.7309 - val_accuracy: 0.6991\n",
      "Epoch 15/50\n",
      "6368/6368 - 217s - loss: 0.5376 - accuracy: 0.7820 - val_loss: 0.7337 - val_accuracy: 0.6975\n",
      "Epoch 16/50\n",
      "6368/6368 - 218s - loss: 0.5370 - accuracy: 0.7822 - val_loss: 0.7308 - val_accuracy: 0.7011\n",
      "Epoch 17/50\n",
      "6368/6368 - 218s - loss: 0.5367 - accuracy: 0.7824 - val_loss: 0.7496 - val_accuracy: 0.6918\n",
      "Epoch 18/50\n",
      "6368/6368 - 218s - loss: 0.5361 - accuracy: 0.7827 - val_loss: 0.7395 - val_accuracy: 0.6971\n",
      "Epoch 19/50\n"
     ]
    }
   ],
   "source": [
    "split_train_val = int(np.floor(len(train_encoder_input) * 0.8))\n",
    "\n",
    "model_checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(\n",
    "    filepath=checkpoint_filepath,\n",
    "    save_weights_only=True,\n",
    "    monitor='val_loss',\n",
    "    mode='auto',\n",
    "    save_best_only=True)\n",
    "\n",
    "model.fit([train_encoder_input[:split_train_val], train_decoder_input[:split_train_val]], \n",
    "          train_decoder_target[:split_train_val],\n",
    "          validation_data=([train_encoder_input[split_train_val:], train_decoder_input[split_train_val:]], \n",
    "          train_decoder_target[split_train_val:]),\n",
    "          epochs=epochs, batch_size=batch_size, verbose=2, callbacks=[model_checkpoint_callback])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.load_weights(checkpoint_filepath)\n",
    "pred = model.predict([test_encoder_input, test_decoder_input])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction horizon = 0\n",
      "accuracy_score = 0.8332998896357982\n",
      "classification_report =               precision    recall  f1-score   support\n",
      "\n",
      "           0     0.7742    0.5304    0.6295     21147\n",
      "           1     0.8450    0.9630    0.9002     98624\n",
      "           2     0.7968    0.5102    0.6221     19767\n",
      "\n",
      "    accuracy                         0.8333    139538\n",
      "   macro avg     0.8053    0.6679    0.7172    139538\n",
      "weighted avg     0.8275    0.8333    0.8198    139538\n",
      "\n",
      "-------------------------------\n",
      "Prediction horizon = 1\n",
      "accuracy_score = 0.7515013831357766\n",
      "classification_report =               precision    recall  f1-score   support\n",
      "\n",
      "           0     0.6780    0.4835    0.5645     27448\n",
      "           1     0.7745    0.9228    0.8422     86605\n",
      "           2     0.6956    0.4582    0.5525     25485\n",
      "\n",
      "    accuracy                         0.7515    139538\n",
      "   macro avg     0.7161    0.6215    0.6530    139538\n",
      "weighted avg     0.7411    0.7515    0.7346    139538\n",
      "\n",
      "-------------------------------\n",
      "Prediction horizon = 2\n",
      "accuracy_score = 0.7735383909759349\n",
      "classification_report =               precision    recall  f1-score   support\n",
      "\n",
      "           0     0.7352    0.6148    0.6696     31915\n",
      "           1     0.7917    0.9089    0.8463     78307\n",
      "           2     0.7469    0.5849    0.6560     29316\n",
      "\n",
      "    accuracy                         0.7735    139538\n",
      "   macro avg     0.7579    0.7028    0.7240    139538\n",
      "weighted avg     0.7694    0.7735    0.7659    139538\n",
      "\n",
      "-------------------------------\n",
      "Prediction horizon = 3\n",
      "accuracy_score = 0.7936977740830454\n",
      "classification_report =               precision    recall  f1-score   support\n",
      "\n",
      "           0     0.7784    0.7181    0.7471     38439\n",
      "           1     0.8124    0.8887    0.8488     66000\n",
      "           2     0.7682    0.6979    0.7314     35099\n",
      "\n",
      "    accuracy                         0.7937    139538\n",
      "   macro avg     0.7864    0.7682    0.7758    139538\n",
      "weighted avg     0.7920    0.7937    0.7913    139538\n",
      "\n",
      "-------------------------------\n",
      "Prediction horizon = 4\n",
      "accuracy_score = 0.8069343118003698\n",
      "classification_report =               precision    recall  f1-score   support\n",
      "\n",
      "           0     0.7895    0.8102    0.7998     47952\n",
      "           1     0.8642    0.8144    0.8386     48050\n",
      "           2     0.7684    0.7950    0.7815     43536\n",
      "\n",
      "    accuracy                         0.8069    139538\n",
      "   macro avg     0.8074    0.8066    0.8066    139538\n",
      "weighted avg     0.8086    0.8069    0.8074    139538\n",
      "\n",
      "-------------------------------\n"
     ]
    }
   ],
   "source": [
    "evaluation_metrics(test_decoder_target, pred)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
